weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming

摘要

Selective weed treatment is a critical step in autonomous crop management asrelated to crop health and yield. However, a key challenge is reliable, andaccurate weed detection to minimize damage to surrounding plants. In thispaper, we present an approach for dense semantic weed classification withmultispectral images collected by a micro aerial vehicle (MAV). We use therecently developed encoder-decoder cascaded Convolutional Neural Network (CNN),Segnet, that infers dense semantic classes while allowing any number of inputimage channels and class balancing with our sugar beet and weed datasets. Toobtain training datasets, we established an experimental field with varyingherbicide levels resulting in field plots containing only either crop or weed,enabling us to use the Normalized Difference Vegetation Index (NDVI) as adistinguishable feature for automatic ground truth generation. We train 6models with different numbers of input channels and condition (fine-tune) it toachieve about 0.8 F1-score and 0.78 Area Under the Curve (AUC) classificationmetrics. For model deployment, an embedded GPU system (Jetson TX2) is testedfor MAV integration. Dataset used in this paper is released to support thecommunity and future work.